MIDMs: Matching Interleaved Diffusion Models for Exemplar-Based Image Translation

We present a novel method for exemplar-based image translation, called matching interleaved diffusion models (MIDMs). Most existing methods this task were formulated as GAN-based matching-then-generation framework. However, in framework, errors induced by the difficulty of semantic across cross-domain, e.g., sketch and photo, can be easily propagated to generation step, which turn leads degenerated results. Motivated recent success models, overcoming shortcomings GANs, we incorporate overcome these limitations. Specifically, formulate diffusion-based matching-and-generation framework that interleaves cross-domain steps latent space iteratively feeding intermediate warp into noising process denoising it generate translated image. In addition, improve reliability process, design confidence-aware using cycle-consistency consider only confident regions during translation. Experimental results show our MIDMs more plausible images than state-of-the-art methods.